inscatolati - Get out of the box! :)

• ISOLINUX is able to boot from the CD directly, without anything to worry about or anything strange to do.

• ISOLINUX, however, is quite limited in respect with what it can do once the CD is booted.

• GRUB is more flexible, and looks nice. Older versions of GRUB had no direct support to boot ISO images. http://www.lrz-muenchen.de/~bernhard/grub-chain-cd.html contain useful info. For 0.93, there was a patch available, which can easily be found on google (by searching something like El-Torito grub support).

  Recent versions of GRUB, however, seems to have a very good support (since march 2004, accordingly to the Changelog). There is plenty of documentation in the info manual (info grub).

• both GRUB and ISOLINUX do support Graphic Splash Screens.

If you are stracing a process and are getting an error like "strace: out of memory", watch out: this is not an error in the application, but an error in strace itself.

This message indicates that strace does not have enough memory to create its own structures to strace your process.

Certain versions of strace, still quite widespread as for year 2005/2006, have a bug in tracing multithreaded applications, which confuse strace about the memory it will need to strace the process.

To verify your application is multithreaded, you can use something like "ps -L aux". If your process as more than one line with the same PID, than it is multithreaded.

Updating strace might solve the problem. Another solution might be to start stracing your process before it spawns other threads. So, don't use the strace '-p' parameter, but strace the process since it starts.

gd

When the cursor is on a local variable, it will jump to its own declaration. It will first search the local scope, and then the rest of the file.

gD

When the cursor is on a global variable, it will jump to its own declaration. Starts searching from line 1 of the file.

gf

Opens the file whose name is under the cursor.

[i

Displays the first line that contains the word under the cursor. This is useful to look at the declarations of functions or variables.

[I

Display all lines that contain the word under the cursor. This is useful to see where a local variable is used and how.

=

Indents the highlighted block, following the rules indicated by the file type.

}

Goes to the end of the current block.

L

To the end of the current screen.

*, #

Search forward/backward for the word under the cursor.

M

Go to the middle of the screen.

set fp?

Prints the name of the program used to format the data.

set ft?

Shows the file type of the open buffer.

gUiw, viwU

Upper cases/lower case the word under the cursor.

set encoding=utf8

Tells vim that the file is utf8 encoded.

ctrl+e, ctrl+y

Moves the current file one line upward or downward.

diffthis

By giving this command in two different buffers, the differences between the buffers are highlighted and colored. It is equivalent to running vimdiff.

Symptoms: customers accessing their own mailboxes report Outlook crashing or the connection being dropped, with a message similar to "Your server has unexpectedly terminated the connection...". You look into the user mailbox, and there doesn't seem to be anything strange. If you look carefully, you should see one or two really small mails.

It seems like Norton Antivirus causes Outlook to crash or the connection being dropped when a user receives a mail without body and no \n after the headers.

The problem has been reported to the CERT as a potential DoS, and Norton seems well aware of that problem. However, no fix is provided.

Two possible solutions:

• configure your mail server to always add a \n after the headers of any received email, even when there is no body.

• disable some Norton Antivirus options. Take a look at this issue on the Symantec Support site and to those instructions for more useful data. The links: Microsoft support and C4 Net support might also contain useful info.

Writing a client for the clamd daemon is usually a matter of a few lines of code: just connect to the daemon socket for each file you need to scan, issue the scanning command that best suit your needs, and that's it most of the times.

To avoid to open/close new connections with clamd continuosly, however, you can issue the SESSION clamd command, which allows a software to issue multiple commands on one, single, socket.

Beware, however, that clamd is able to recognize one, and only one command per packets it receives. If, after issueing the SESSION and STREAM command, for example, your client hangs, it's probably because the Linux Kernel joined the two different commands in one, single, packet.

The only solution we know about is to insert a sleep(1); between the write("SESSION"... and the write("STREAM"... command, hoping that a 1 second delay would be enough to avoid the nagle algorithm.

Note that we know no way to disable the nagle algorithm on Unix stream sockets, and afaik, there is no way to avoid the kernel joining the buffers but a small delay.

The problem is known to the clamav developers, but no solution has been proposed yet, mainly to avoid breaking compatibility with older clients.

If you run the "mount" command alone, you should get the list of the mounted partitions, something like:

/dev/sda1 on / type ext3 (rw,errors=remount-ro) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw)

Sometimes, especially when the root partition contains some errors, or has been mounted/remounted read only, the output of mount is screwd up: read-only partitions are shown as being read-write, unmounted partitions are still shown as mounted, and so on.

The problem is due to the fact that mount uses the file /etc/mtab for book keeping, writing there whenever a partition is mounted, unmounted, or remounted with different options.

If /etc/mtab is read only, for example because the root partition is read only, mount and many system tools may get confused about the status of the partitions, and may not be able to update their status. Symptoms are: partitions being unmounted without errors and still being shown by the mount command, read-only partitions shown as read-write, and all kind of inconsistencies...

To get a clean and correct view of the mounted partitions, you should take a look to the file /proc/mounts, with something like "cat /proc/mounts". That file shows the status of the partitions as seen by the kernel, and is correct most of the times.

Note, however, that on recent kernels (2.6.x and greater), every process may have a different "view" of which file systems are mounted and unmounted. If the situation gets that much confusing, one way to understand if a partition is read-write or read-only, is just to try to "touch" a simple file... have fun :)

If you are planning to make a TLA archive available via WEB, either in read-write mode using WebDAV or in read-only mode using the plain old http protocol, you should create your tla archives with the command:

$ tla make-archive -l ...

(note the -l).

However, if you have an older archive that you now want to make publicly available using the http protocol, you need to ssh/telnet/ftp on the server hosting your archive, create an empty file http-blows in the =meta-info directory, and then run the command:

$ tla archive-fixup -A archivename

Where archivename is the usual my.mail@domain--nameofarchive, and where -A ... can be omitted in case you are working on your my-default-archive.

In short, something like:

$ ssh my-remote-server-hosting-tla server $ cd /my/archive server $ touch =meta-info/http-blows server $ logout $ tla archive-fixup -A my@email--my-archive

solved the problem for me :)

If you are seeing this error when trying to access a web TLA archive, it probably means that the archive was created without the '-l' options to the make-archive command, or, for some reasons, no .listing files were created.

In this case, the only way to fix the archive is to ask the archive administrator, or someone with write permissions on the archive, to create the 'http-blows' file and to run a 'tla archive-fixup' command.

For more details, please take a look at http://notes.inscatolati.net/[en]/software[en]/tla[en]/index.html#7

When writing an internal anchor, something like <a name="nameofanchor">, either use the form:

<a name="nameofanchor"> or <a name="nameofanchor"></a>

Some browsers, notably some versions of firefox, did not understand anchors like:

<a name="nameofanchor" />

as usable with XML or XHTML. Indicating the correct DOCTYPE might also solve the problem :), even if this solution has not been tested...

When creating a form, if the "method" attribute contains leading spaces, some versions of Internet Explorer will submit the form using the GET method, regardless of the value of the attribute. As for 2005/2006, all recent versions of IE exhibit this behavior. As an example:

<form method="GET" ... # correct, GET method used <form method="POST" ... # correct, POST method used <form method=" POST" ... # wrong, GET method used

Note that other browsers, like firefox, ignore the leading spaces and will correctly send data using the POST method.

t

Open a link in a new tab.

>

Move to the next tab.

<

Move to the previous tab.

ctrl+t

Modify the text box using your own EDITOR, as defined by the EDITOR environment variable (default: vi).

L

List actions that can be performed on a given element. Most useful on javascriptish pages :)

Ok, to set certaion options automatically for a given file, you can simply use something like:

vi:ts=2:

where vi: and the final : are mandatory, while ft=php is just a simple option that instructs vi about the format of the file being edited.

Usually, this "string" is put as a "comment" inside the file being edited. In some programming languages, the above option would look something like:

in Perl: # vi:ts=2: in PHP: // vi:ts=2:

Watch out, however, that if you want to put "other characters" after the special command, you need to specify the "set" command, with something like:

in PHP: /* vi:set ts=2: */ enclosing the command in square brackets: # [ vi:set ts=2: ]

To have complete help about the above options, you can use ":help vi:" or ":help modeline".

On recent Linux kernels, tcpdump can listen on multiple interfaces.

In order to do so, just specify the 'any' virtual interface with something like:

# tcpdump -i any

When the 'any' interface parameter is specified, interfaces are not put into promiscuos mode.

To manually set interfaces into promiscuos mode, just use something like:

# ifconfig eth0 promisc

or

# ip link set eth0 promisc on

Note that we don't know any way to specify selectively a list of interfaces to listen on, and we don't know any way to have an indication of the name of the interface on which a given packet was captured.

A workaround could be the '-e' parameter, to have link-level headers dumped. Note, however, that link-level headers may easily be spoofed or just wrong.

Ok, while sniffing traffic, some options might actually be useful:

when inspecting the content of the packets...

use something like '-X -s 8192 -i eth0', where '-X' indicates to print packets both in HEX and ASCII, '-s 8192' increases the number of bytes tcpdump will actually inspect, and '-i eth0' indicates to listen on 'eth0'. Note that if you want to print the content of the whole packet, with '-s' you need to specify a value higher than the MTU of the interface. You can look at the MTU of your interface by using 'ifconfig eth0' or something like 'ip link show dev eth0'.

when checking routing/firewalling/nat problems...

use the '-e' parameter, to look at the link-level headers. Note that if we do not consider NAT, all IP packets will always have as src ip the ip address of the sending machine, and as dst ip the ip address of the final destination.

Packets that need to pass a router/gateway/firewall... will have, as dst IP, the IP address of the final destination. The packet, however, will go to the router thanks to link-level addressing, which, on ethernet, will cause the packet to have the MAC address of the router as the address of the recipient.

when looking for connectivity problems with particular networks/addresses/...

use the '-vvv' parameter, and have a careful look to all the headers printed by tcpdump. Take special care in checking ICMP packets (fragmentation requested, administratively prohibited, ...), fragmentation, the TTL, and various IP/TCP options that might be set on the packet.

Also, remember to write a filter to isolate packets coming from the network you are inspecting. Watch out, however, that certain network errors might actually come from routers and/or other IP addresses than those you are filtering, so watch out not to filter ICMP packets and not to be too strict with your filters. Something like:

# tcpdump -n -vvv 'net xx.xx.xx.xx/24 or icmp'

Should work as expected.

• greatly slow tcpdump down

• create a mess if no filter has been given, or if you are inspecting DNS packets. Without '-n' ip addresses will be transformed into hostnames. Afaik, this will require DNS packets to be sent out to your own dns 'sometimes' (depending on the resolver cache), confusing the output a lot.

Manually setting the MTU allows you to force the kernel to send smaller packets regardless of the media being used or protocols like Path MTU discovery or similar.

You can set the MTU either for a whole interface, using something like 'ifconfig eth0 mtu 200' or 'ip link set eth0 mtu 200', or for a single path with something like: 'ip route add 192.168.0.0/24 via 10.0.0.1 dev eth0 mtu 200'. In this case, you need to have the 'iproute2' package installed on your system.

You may as well change the 'advertised' MSS to ask remote ends to send you smaller or bigger packets. To change the mss for a single route, just use something like: 'ip route add 192.168.0.0/24 via 10.0.0.1 dev eth0 advmss 200' if you have the iproute2 package installed, or 'route add -net 192.168.0.0 netmask 255.255.255.0 gw 10.0.0.1 mss 200'.

Ok, so you just wrote your wonderful daemon that listens on a given sockets and provides some nice service over the network?

With manual file descriptors handling (using open, write, read, ...) it is easy to make errors in handling buffers and network conditions correctly.

If you had reports about people not being able to use your software from certain internet locations, or you are ready to release a new version of your software, you should check the correct behavior of network related routines with different MTU and packet sizes. Small packets can easily trigger buffering errors, or make some of the assumptions you made about your code miserably fall over below your floor.

One easy way to check the behavior with different MTUs is just to set the loopback MTU to a different size, and then perform some tests on the application by connecting to 127.0.0.1.

To know more about how to set the MTU of an interface, please refer to http://notes.inscatolati.net/[en]/system[en]/networking[en]/index.html#15.

One easy way to have the list of ports open on a Linux box is to use the 'netstat -nlp' command, 'netstat -ntlp' just for TCP, or 'netstat -nulp' for just UDP, where:

-n

tells netstat not to resolve ip addresses/port numbers into hostnames/port names.

-l

tells netstat to show only sockets waiting for connections (in listen state).

-p

tells netstat to show the pid of the involved processes.

-t

tells netstat to show the pid of the involved processes.

Sometimes, however, netstat will not show the pid of a given process, and will show a '-' instead. That does not mean there is 'no process' associated with the listening socket, or that your box has been hacked as you can read on some messages on various mailing lists.

Most of the times, those ports have been opened by the linux kernel directly, and that is why 'there is no process' associated. Are you using khttpd? nfs? sun rpc? Most of the times, those ports are related to the 'portmapper' and protocols based on sun rpc. To see which ports are related to which service, supposing that rpc is the reason, you can use 'rpcinfo -p'. By running 'rpcinfo -p' on my notebook, I can see that port '2049', not bound to any process accordingly to 'netstat', is the 'nfs' server. By running 'ps aux', I can see that something like '[nfsd]' is running (kernel NFS daemon), and with 'lsmod' I can see that the 'nfsd' module is running. If I go to /proc/2839, which is the PID of the nfsd process, I can see that the 'exe' symlink does not point anywhere: a good indication that 'nfsd' is not a real program, but a component of the kernel itself.

By using:

# netstat -s

it is possible to obtain nice-looking general purpose networking statistics, like the number of tcp connections established, failed, the number of ip packets sent, the number of packets with bad checksums, the number of segments retransmitted, and so on...

Just run:

# apt-get install linuxlogo

and follow the instructions. Tipically, you will have to modify the file /etc/inittab and add something like:

2:23:respawn:/sbin/getty -f /etc/issue.linuxlogo 38400 tty2

If you want to change the partition table of a given device on a running system, without rebooting, just go on and modify the partition table, paying attention not to overwrite or shrink any used partition.

Once the partition table has been changed, you need to ask the kernel to reload the partition table and update the devices accordingly. To do so, you can run the command:

# sfdisk -R /dev/sda

where sda is the name of the device where the partition table was changed. If you get the error:

BLKRRPART: Device or resource busy

it means that /dev/sda, or the device you are using, has partitions mounted on it, and the kernel was unable to read the new table. If you want to use the new partition table without rebooting, you need to umount all the old partitions of /dev/sda and then re-run 'sfdisk -R'.

If 'sfdisk -R' returns no error, it means the new partition table was read and that the devices /dev/sda1, /dev/sda2... are now relative to the new partition table you just updated.

• if you run:

  $ cat /proc/partitions

  you get the list of all partitions, as seen by the kernel. This list is usually generated at boot time and updated every time a command issues the BLKRRPART ioctl (sfdisk -R to issue the ioctl manually). This list also includes virtual devices, like those created with the LVM tools and the device-mapper.

  Note that if the partition table was changed, and the system not rebooted, or no BLKRRPART issued or issued with other partitions mounted, /proc/partitions may not reflect the content of the partition tables of the disk. Note also that if the kernel did not detect some partition table or the content of some devices, those partitions might not be available from /proc/partitions.

  Note also that any user can read the content of /proc/partitions, and that all devices in /dev representing partitions will reflect the content of this file.

• you can run:

  # fdisk -l

  or

  # sfdisk -l

  both commands will read the partition table directly from the disk. sfdisk is a very low-level tool, so, the output may slightly differ (sfdisk often shows even unused partitions).

  Note that the above commands can only be run by privileged users. Note also that the output indicates the exact content of the disk, which might not reflect the 'vision' of the kernel. If the partition table was zeroed with a 'dd' without rebooting, or the partition table was changed with a simple fdisk, with 'fdisk -l' you will see the new partition table. However, the devices in /dev, like /dev/sda1 or /dev/hdc2, will refer to the partition table as seen by the kernel, and as readable by the /proc/partitions file.

Ok, symptoms:

• Your script dies with an error similar to the following:

  stream /dev/conslog is not writable at ... console is not writable at ... no connection to syslog available ...

• If you run your script as 'root', everything works perfectly, while as a normal user syslog does not seem to work... no! users should be allowed to log messages! it's not a privileged operation! don't ever think about that!

• 'logger' or similar commands seems to work and log fine, without troubles, both as users and administrators... I used something like:

  $ logger -t test -p local0.notice 'test' $ su # logger -t test -p local0.notice 'test'

• make sure syslog is properly configured and running and that devices/unix sockets have the correct permissions, by checking that the above command ('logger ...') works correctly.

• make sure your code calls the 'openlog' functions once and only once. On my perl 5.8.4 Linux System, I had two different modules calling the 'openlog' function... and well, as 'root', everything worked just fine, but as soon as I switched to an unprivileged user, the second 'openlog' caused the error described.

Ever had an error like:

$ ./my-perl-script Can't locate ... in @INC (@INC contains: ...) at ... line ... . BEGIN failed--compilation aborted at ... line ... .

Well, this often happens when you have a directory with your perl scripts and its own modules, which you have not installed yet.

A good solution is usually to go in the lib/ directory of the tarball, or the directory containing the name of the missing module. For example, if the message above complains about missing RBC/XML.pm, with something like:

Can't locate RBC/XML.pm in @INC (@INC ...

You can simply:

$ find . -type d -name 'RBC' ./lib/RBC $ cd lib $ ../my-perl-script

Well, if you can't change directory, or you get another error about another missing library, you can simply use the PERL5LIB or PERLLIB environment variable, with something like:

$ PERL5LIB=/home/.../lib/ ./my-perl-script or $ export PERL5LIB=/home/.../lib/ $ ./my-perl-script

Note that if you use an absolute path in PERL5LIB, you can run my-perl-script from anywhere on your file system. Instead of PERL5LIB you can always use PERLLIB, or the -I parameter to the perl executable, with something like:

$ perl -I/home/.../lib/ ./my-perl-script

PERLLIB and PERL5LIB are ignored in case the script is somewhat privileged. To know more about all of this, just run:

$ man perlrun

and search '-I' or 'PERL5LIB'.

In kernel 2.4 and first 2.6, raid partitions aka md devices were not partitionable. Even today, most distributions do not support md device partitioning.

There are not many solutions to this problem, either:

• create a single raid device for the whole disk (for example, between /dev/sda and /dev/sdb), create a file system on that single partition, and live with it (mkfs.ext3 /dev/md0, where md0 is made of /dev/sda and /dev/sdb). This should have no problems, beshide that it is not very sane to put a complete system on a single partition...

• create a single raid device for the whole disk (for example, between /dev/sda and /dev/sdb), and use the device mapper (aka lvm) to create logical volumes (aka, partitions). This works, but you cannot boot from a logical volume... neither lilo nor grub support that.

• first create the partitions, and then create the raid devices. For example, you could create a small /dev/sda1 and /dev/sdb1, and a md0 device with the /boot directory on it. Then, create /dev/sda2, /dev/sdb2, /dev/sda3, /dev/sdb3 as needed and raid them separately... this solution works well, but:

  • you have to pay attention when installing the boot loader. Since only partitions are on 'raid', everything out of the partitions, like the MBR (Master boot record) is not in 'raid'. If you simply install grub or lilo as usual, at the first failure, after replacing a disk, you will likely end up with an unbootable system.

  • the solution is not very flexible. If you need to create a new partition, you have to create a new raid device, and so on... if you need to change something, it's hard to do...

  • in case of a disk failure, you will probably end up with many software raid devices not working anymore, and with the kernel having to realize that all or some of them have failed...

• Well, simply create two partitions, /dev/sda1 and /dev/sda2, raid /dev/sda1 with /dev/sdb1, and then use /dev/sda2 in raid with /dev/sdb2 as an phisica volume for LVM. That way, you can boot the system, and create new partitions simply by using the LVM.

With kernel 2.6 and a recent version of mdadm, you can also use the option --auto=mdp to create a partitionable raid volume. One more solution could be to use the EVMS, still not part of the official linux kernel.

If you use LVM over RAID 5 on a 2.4 kernels, you might easily have errors like:

raid5: switching cache buffer size, 4096 --> 1024 raid5: switching cache buffer size, 1024 --> 4096 [...]

These messages are repeated almost everytime there is a disk access, flooding both your console, your logs, and degrading performance.

A quick and easy hack would be to create the filesystem with a block size of 4096. In the case of ext3, something like:

# mkfs.ext2 -b 4096 /dev/mapper/vg00-lv00

With this option, you will see your logs being flooded only when you are using low level tools, like 'vgs', 'lvs', 'tune2fs', and so on...

With other file systems, there are equivalent options to set the block size to 4096 bytes...

Ok, let's say you have a software RAID 1 /dev/md0 device made of two partitions on two different scsi disks, /dev/sda1 and /dev/sdb1.

Let's say you just had a major hardware failure, and for one reason or another, some data was corrupted on the first device, while some other data was corrupted on the second device.

One easy way to try to recover data is to mount /dev/sda1 and /dev/sdb1 individually, recover data, and then, eventually, put them back into the raid.

Doing so, is quite easy:

• if the raid device is still mounted, umount it immediately, with something like 'umount /mount/point/of/raid/device'. You can see the mount point of the md device with something like 'mount |grep md0' or 'cat /proc/mounts |grep md0' (see note mount, read only partitions, and wierd output).

• stop the raid device, with something like:

  # mdadm --misc --stop /dev/md0

  or mark it read only, with:

  # mdadm --misc --readonly /dev/md0

• simply mount the devices independently, using:

  # mount -o ro /dev/sda1 /mnt/sda1 # mount -o ro /dev/sda2 /mnt/sda2

  making sure the /mnt/sda1 and /mnt/sda2 directory exist.

• You can now work on /mnt/sda1 or /mnt/sda2 without problems (...).

• if you later decide that the content of one of the partitions is good, you can start the array in degraded mode with --assemble, or mark /dev/sda2 (for example) as failed, and then simply replace it ...

Ok, so... you are switching from a non-LVM system to using LVM... you have your /dev/sdb, and want to turn it in a Physical Volume to add to a simple Volume Group.

You try with a simple:

# pvcreate /dev/sdb

and you get an error:

Can't open /dev/sdb exclusively. Mounted filesystem?

you check the mount option, with something like:

mount |grep /dev/sdb

and nothing appears... (well, if something appears, just run umount all mounted partitions, and try one more time). So: pvcreate fails, reporting 'Mounted filesystems?', but no file systems are mounted.

The solution is quite simple: as long as the kernel sees partitions on the device, pvcreate will not be able to lock it to create a physical volume... (this is still true on a 2.6.16 with lvm2 2.02.05). So, remove the partitions (with cfdisk/sfdisk/fdisk, whatever you want, or dd if=/dev/zero of=/dev/sdb size=512 count=1) run 'sfdisk -R' and you should be fine... run pvcreate, and you will have no more errors...

# dd if=/dev/zero of=/dev/sdb size=512 count=1 # sfdisk -R # pvcreate /dev/sdb

To backup a partition table, you can simply run:

# sfdisk -d /dev/sda > /home/backup.file

To restore the backup, which means, to repartition the disk as the dump you just generated, you can simply run something like:

# sfdisk /dev/sda < /home/backup.file

Note that the above commands do not save/restore the full MBR: if a boot loader or something similar is installed, its own code will be lost... you may have better luck by using dd, but watchout with extended partitions...

On Windows 2000, if you boot the PC with your poor, simple, 56k modem turned off... you loose the ability to use the modem!!

Solutions: reinstall the driver of the modem, or simply reboot your pc with the modem turned on. This time, it will properly work...

To backup the partition table using dd, simply run:

# dd if=/dev/sda of=/tmp/backup.pt bs=512 count=1

where /dev/sda is the device of which you want to backup the partition table, /tmp/backup.pt is the name of the file where you want your partition to be stored, and bs=512, count=1, tells dd to copy 1 sector of 512 bytes from the beginning of the disk...

This method will backup the full MBR, both the partition table and a small fragment of code used to boot your system in front of the partition table. However, IT WILL NOT BACKUP EXTENDED PARTITIONS!

This means that if you have /dev/sda5, /dev/sda6 or greater... the partition table records about those partitions will be lost. A better solution would be to save both the first sector of the disk using dd, and then the complete backup of the partition table using 'sfdisk -d' (take a look at http://notes.inscatolati.net/system[en]/storage[en]/index.html#29).

So, you have just bought 10/15 bleeding edge scsi disks, of the same type, brand, and of the same kind, and you want to create a partition table on each of them?

A simple way would just be to create a partition table on the first device (sda), with cfdisk, fdisk or whatever you like, and then backup the partition using:

# sfdisk -d /dev/sda > /tmp/backup.pt

Now, for each device, you can simply run:

# sfdisk /dev/sdb < /tmp/backup.pt

In short, something like:

# cfdisk /dev/sda # sfdisk -d /tmp/backup.pt # for dev in /dev/sd{b,c,d,e,f,g,h,i,j,k}; do \ > sfdisk $dev < /tmp/backup.pt; done

should be enough...

At least twice in my life I've been surprised by grep either not matching what it was supposed to, or matching too much.

Both times it turned out to be a problem with the locale configured on the system. Yes, locale settings influence things like what are to be considered whitespaces, letters, and so on. grep and many other matching libraries keep that in consideration, and change behavior accordingly.

While this is kind of expected, if the locale configuration is screwed for the language being used, grep (and many other tools...) may really get confused.

Check the locale configurations, and try with things like:

LC_ALL=C grep ...

to see if the problem goes away or grep changes behavior. If it does change, well... you know for sure the problem is with the locales. Good luck with the debugging :)

Either:

• Check the documentation of the module one more time, you are using it the wrong way.

• If you wrote the module yourself, you forgot to add the line:

  package Full::Name::of::module::XXX;

On any linux systems, setlocale will fail unless you have the data for the selected locale compiled and available.

The symptoms range from:

• setlocale() returning NULL

• scripts returning errors like:

  perl: warning: Setting locale failed.

• web interfaces like imp, squirrel, or horde ignoring the language and the settings you selected

In Debian, to select the locales you want compiled and available, you need to use the command:

% dpkg-reconfigure -plow locales

This script will generate a file /etc/locale.gen listing all the locales that you are interested into. It will then call /usr/sbin/update-locale and /usr/sbin/locale-gen, which is the real script that takes care of compiling the locale files by running something like:

% localedef -i en_EN -c -f ISO-8859-15 -A /usr/share/locale/locale.alias en_EN

localedef, at least in Debian, will update the files in /usr/lib/locale/*

Let's say you have a LDAP database served by openldap. Let's say that your applications keep querying this database to provide authentication / addressbook / whatever. At a certain point in time, slapd starts telling you that a certain entry does not exist when you try to fetch it directly, but, it does show up when doing a 'sub' search or using slapcat.

This is usually a symptom that the index of the ldap database is corrupted. You can regenerate it by using 'slapindex' as root. After running it, remember that if you are not running slapd as root, you should fix the privileges of the files (chown slapd *).

You might also want to regenerate the database, something like slapcat > /tmp/backup; rm -rf database/files; slapadd < /tmp/backup or similar.

initrds created with mkinitramfs, mkinitrd, yaird or any other tool can sometime contain errors that make your system un-bootable, or that output error during the boot process. It is usually a pain to debug those errors, as... you often don't have a shell, needed softwares are missing from the initrd, ...

Generally, there are two approaches that can be easily used to debug an initrd:

• uncompress the initrd, and have a peek in the scripts to see what's being done and what it is doing when the init process stops (and why) - always works.

• at boot time, have the initrd output some debugging lines, or get a prompt to try to manually understand what's going wrong or why the commands are failing. Using this method requires support from the tool used to create the initrd, so it won't be discussed in this note.

So, to access the content of an initrd, you need to run a few commands depending on the format of the initrd itself. Nowdays, most initrds are either gzip-compressed cpio files, cramfs or other more or less esotheric file systems. You can start with something like:

% file -Ls /boot/initrd.img

If you are lucky, it will be a cramfs:

/boot/initrd.img-2.6.8-3-686-smp: Linux Compressed ROM File System data, [...]

just mount it with something like:

% mount -o loop /boot/initrd.img /mnt/whatever

If you are a bit less lucky, it will be gzip compressed:

/boot/initrd.img: gzip compressed data, from Unix,

Start by just uncompressing it:

% gzip -cd < /boot/initrd.img > /tmp/initrd.uncompressed

Repeat the "file" command above against /tmp/initrd.uncompressed. If you are lucky, again, it will be a filesystem. Just mount it with the same "mount -o loop..." as above. If you are a bit less lucky, you will see something like:

$ file -sL /tmp/initrd.uncompressed /tmp/initrd.uncompressed: ASCII cpio archive (SVR4 with no CRC)

which means that the initrd is a simple cpio archive. Uncompress it with something like:

$ cpio --extract --make-directories < /tmp/initrd.uncompressed

Now, you can start your debugging by either looking into /linuxrc, /init, or /sbin/init. You can obviously modify the filesystem, and reverse the steps to create a new initrd to test. Good luck!

mkinitramfs allows you to specify some parameters on the LILO or GRUB prompt to easily (sure) debug problems. Most useful parameters are probably:

• debug, to have all the shell scripts on the initrd run with the -x parameter, and the output logged in /tmp/initramfs.debug on the ramdisk. To have the output sent to stdout, specify something like debug=/dev/console.

• break, to have the initrd return a shell prompt. If no parameters are specified, the prompt will be returned when most convenient to the initrd (at current time, just before mounting the root filesystem - premount). Otherwise, you can specify something like break=whatever, to ask the initrd to stop at exactly the specified step. Steps currently defined by mkinitramfs are: top, modules, premount, mount, bottom and init.

• blacklist, if you suspect a module is causing problems to your hardware. Specify something like blacklist="module1 module2" to have module1 and module2 not loaded at boot time.

As I use grub on my system, to specify those parameter at boot time, I usually press 'e' on the line I usually boot from. Once there, I modify the line:

kernel /vmlinuz root=/dev/mapper/root ro

to have the options I need:

kernel /vmlinuz root=/dev/mapper/root ro break=top

and finally press 'b' to get the system booted with the specified parameters.